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I  SUMMARY 

1  A  decomposition  principle  for  linear  programming  is  pre- 

j  sented.  The  technique  may  be  viewed  as  a  dual  of  the  Dantzig 

Wolfe  decomposition  principle  for  linear  programs  [  7  ]  .  The  pro- 

I  gram  matrix  in  what  we  may  call  the  basic  problem  is  considered  as 

I  having  many  (an  infinite  number  of)  columns.  As  in  the  Dantzig 

Wolfe  treatment,  one  visualizes  a  basic  problem,  in  primal  form, 

I  where,  however  the  set  of  permissible  columns  is  not  finite,  as  in 

I  the  usual  primal  form,  but  is  a  given  convex  polyhedron.  The  basic 

problem  is  solved  by  the  modified  simplex  method  [ll]  ,  but  at 
I  each  iteration  the  column  to  enter  the  current  basis  emerges  as  the 

j  solution  to  an  auxiliary  linear  program  and  is,  in  fact,  an  extreme 

point  of  the  given  convex  polyhedron. 
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A  DUAL  DECOMPOSITION  PRINCIPLE 


I.  Introduction  and  Notation 

One  of  the  main  obstacles  to  the  full  application  of 
linear  programming  is  the  inability  of  most  current  computational 
methods  to  cope  with  problems  yielding  program  matrices  of  large 
order.  However,  many  of  these  problems  have  program  matrices  which 
exhibit  special  structures,  e.g.  diagonal  or  block  triangular  and 
which  are  very  sparse  in  non-zero  elements.  A  large  order  trans¬ 
portation  problem  is  an  example.  The  size  of  the  problems  we  have 
in  mind  make  it  infeasible  to  attempt  to  solve  them  by  a  general 
technique  even  on  the  most  modern  digital  computers. 

Much  current  research  is  being  directed  toward  developing 
special  techniques  taking  advantage  of  these  matrix  structures  in 
order  to  facilitate  solution  of  such  problems  as:  (i)  traffic  flow 
problems  and  network  problems  [^1,  4,  12^  ,  (ii)  gasoline  blending 
problems  [[3I  ,  (iii)  communications  problems  ,  (iv)  ware-housing 

problems  [2]  ,  (v)  production  planning  problems  [5,  I3]  ,  (vi)  the 
caterer  problemf^sj  ,  and  (vii)  von  Neumann's  model  of  a  constantly 
expanding  economy  [14]. 

One  recent  technique  for  handling  certain  intrinsically 
large  order  problems  is  embodied  in  the  Dantzig  -  Wolfe  decomposition 
principle.  We  shall  review  the  basic  notions  of  their  principle. 
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taking  the  liberty  of  using  our  own  notation.  We  then  develop  our 
dual  decomposition  principle,  so  as  to  compare  the  two.  Dantzig  and 
Wolfe  apply  their  technique  to  some  problems  of  block  triangular  form. 

Consider  the  general  linear  programming  problem  in  the  form: 
(1)  max  Xq 

where  XqPo  +  X  =  'fe 

A  X  =  b,  X  >  0  . 

We  think  of  A  as  being  of  large  order,  but  fairly  easy  to  handle. 

For  example,  we  may  have: 


A  = 


Ai  0  .  . 

0  A2 


.  0 


.  •  0  .  An 


where,  without  any  coupling  constraints,  A  X  =  b  would  break  down  into: 
Ai  X^^bi,  i=  1,2,  ...,  T  • 

In  general,  our  aim  is  to  treat  efficiently  the  case  where  A  has 
block's  of  zeros,  variously  placed. 

We  consider  one  "iteration"  of  the  Dantzig  -  Wolfe  algorithm. 
We  assume,  which  Dantzig  and  Wolfe  do  not,  that  the  convex  set 


|x|AX=b,  X>0j  is  bounded.*  So,  in  our  discussion  of  their 
algorithm  we  will  not  consider  the  possibility  of  homogeneous  solutions 
to  (1)  which  may  arise  when  the  above  set  is  not  bounded. 

We  may  pose  the  problem  in  the  form: 

(2)  max  Xq 

A 

where  Xq  Pq  +  P  =  b  , 

where  P  belongs  to  C  =  |p|p  =  AX;  AX=b,  X^O^  . 

Dantzig  and  Wolfe  assume  that  one  has  a  feasible  solution 
(5^,  X),  and  in  fact  points  Xi,  X2,  .  .  .  >  such  that: 


(3) 

m 

X  =  ^  Xi  ; 

i  =  1 

i  =  1 

> 

IV 

0 

where 

the 

Xi 

are  extreme  points  of  | 

■x|  A  X  -  b,  X> 

“]• 

Thus , 

if 

Pi  = 

A  X^,  then: 

(4) 

-  Pi- 

i  =  1 


One  may  then  pose  the  problem  in  the  form: 

(5)  max  Xq 

m  ^ 

where  Xq  Pq  +  X  P  +  =  b, 

i  =  1 
m 

X  +  y  -  1  :  A.A.io, 

i  =  1 

*  This  is  readily  realized  by  the  technical  device  of  adding  the 

constraint  Xi  +  .  .  ,  +  X^  $  K  where  K  is  sufficiently  large. 


where  P  is  any  point  in  C.  We  thus  think  of  (5)  as  the  "basic 
problem. " 

One  further  assumes  that  the  matrix: 


(6) 


is  non- singular. 

We  shall  now  review  one  "iteration"  of  the  Dantzig  - 
Wolfe  technique.  If  we  write: 


(7) 


then  TT  ■=•  j  is  the  set  of  "simplex  multipliers"  associated  with 
the  "basic  feasible  solution"  obtained  by  setting  Xq  » 
and  A  »  0  in  (5).  Hence H  is  the  unique  point  satisfying  the  con¬ 
ditions: 


T 

PO  "TT,  =1 
T 

+p=0.  >i=l>2, 

We  assume  that  the  inverse  (7)  has  been  confuted. 

^  A 

The  solution  (Xq,  X)  is  optimal  if: 

(8)  77/  P  >  0 


for  all  pec. 


This  suggests  that  we  consider  the  auxiliary  problems: 
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(9)  min  X 
where  A  X  =  b,  X>0, 

_  T 

where  Y  ^  k  TT, 

If  X  is  an  (extreme  point)  solution  to  this  auxiliary 
problem,  then  >  X)  is  the  optimal  solution  if  ^  ^  O 

If  not,  then 

(10)  P  =  A  X 

is  introduced  into  the  basis  B  in  the  usual  simplex  method  manner 
and  the  iteration  is  then  completed. 

In  summary,  with  reference  to  the  ’’basic  problem”  (5),  one 
examines,  in  the  usual  simplex  method  way,  the  "relative  costs”  of 
each  "column”  P,  using  the  multiplier  (if,  ^  (3^  to  ascertain  whether 
or  not  optimality  is  reached,  as  indicated  by  ^  for 

all  such  columns.  However,  here  this  means  examining  all  the  points 
P  of  a  convex  polyhedron  C,  suggesting  the  auxiliary  problem. 

Now  with  reference  to  (9) ,  notice  that  at  each  iteration 
only  the  functional  in  the  auxiliary  problem  changes.  When  one  has 
a  new  functional,  and  wishes  to  solve  (9)  one  starts  from  the  optimal 
solution  to  the  previous  auxiliary  problem  and  continues  on  (possibly 
in  a  few  iterations)  to  the  minimum  of  the  new  functional. 

The  Dual  Algorithm 

Both  our  algorithm  and  the  Dantzig  -  Wolfe  algorithm  agree 
in  (i)  having  the  "basic  problem”  in  the  form  (5),  (ii)  having  the 
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non-singular,  current  basis  matrix  in  the  form  (6);  (iii)  assuming 
that  the  current  inverse,  given  by  (7),  has  been  computed;  and  (iv) 
having  the  same  convex  set  C. 

Our  algorithm,  which  is  in  one  sense,  a  dual  of  the  Dantzig- 
Wolfe  technique,  is  applied  to  the  dual  of  the  "basic  problem"  de¬ 
fined  by  (5),  namely; 

(11)  min  +^) 

T 

where  ^0  ^ 

0  ,  j  =  1,  2,  .  .  .  .  ,m 
V^IT,  >  0,  for  all  P  £  C  . 

We  of  course  assume  that  both  (11)  and  (1)  have  solutions. 

At  the  start  of  one  iteration  we  also  assume,  as  so  Dantzig 
and  Wolfe,  that  we  have  a  starting  feasible  solution  {TFij  (5  )  to  (11) 
and  a  74  such  that  \^1\2  ~  ^  • 

As  was  mentioned  previously,  we  assume  that  the  convex  set 
j  A  X  =  b,  X  >  oj  is  bounded.  Dantzig  and  Wolfe  do  not  make  this 
assumption  which  is  an  essential  part  of  the  derivation  of  our  auxiliary 
problem. 

Our  algorithm  first  of  all  differs  from  the  Dantzig  -  Wolfe 
technique  in  assuming  that  their  optimality  criterion,  given  by  (8) , 
is  satisfied.  The  optimality  criterion  for  our  dual  technique  is 
given  by  the  following: 
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^  min  ^  ^ 

Theorem:  Let  r  be  such  that  Aw  =  i  |  /\  J 

If  AriO,  then  (Xq,  X)  is  optimal. 

So  at  the  start  of  each  iteration,  we  coitqjute  A  i;,  jf 

A  r  >0,  then  we  are  finished.  If  A  r<  0,  then  the  column 


is  the  VGO  (vector  going  out  of  the  current  basis)  . 

In  order  to  determine  a  vector  coming  into  the 

basis,  VCI,  we  solve  the  auxiliary  problem  given  by: 

(12)  minJ^yTy 

where  Ay-  7^^  «  0 

and  y'^d  P^)  +  J  -I  ;  y  >  0,  ”>[  >  0  . 

The  solution  of  (12)  yields  P  :=  JJ  which  is  the  VCI. 

^  ' 

Expressing  F  in  terms  of  the  basis,  we  get: 


(13) 


=  P 


0 


The  new  the  A  ^  V  i 


are  computed  from 


the  formulas: 


and 


where  p. 


/  p\^Ip^ 


V 


A,  -  A.  - 

-  X 
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This  completes  one  iteration. 

Now,  unlike  the  Dantzig  -  Wolfe  Technique  our  auxiliary 
problems  differ  not  only  in  the  functional  but  also  in  the  addition¬ 
al  constraint.  So,  if  we  solve  (12),  we  will  not  be  able  to  take 
maximum  advantage  of  the  previous  confutations  as  do  Dantzig  and 
Wolfe  when  they  solve  their  auxiliary  problems.  This  suggests  that 
we  solve  the  dual  of  (12)  as  a  primal  problem.  The  dual  of  (12)  is 
given  by 

max  0 

(14)  where  -  e(A'^  p’^)  <  T^'^TT, 

b^TT^  -  0  if 

We  solve  (14)  by  the  modified  simplex  technique  |[ll^  .  We  will 
now  show  that  by  means  of  one  artificial  iteration  we  can  avoid 
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the  problem  caused  by  the  additional  constraint  and  hence  start  from 
the  optimum  solution  to  the  previous  problem  and  continue  on  to  the 
maximum  of  the  new  functional  thereby  taking  maximum  advantage  of  the 
previous  iteration. 

At  the  start  of  one  iteration  in  the  solution  of  (14)  we 
assume  that  we  have  a  basis  By  The  columns  of  are  selected 

from  the  matrix: 


where 

I  0 

0 

(14). 

Let  Bl 

(15) 

I  0  -a""^ 

0  1  - 

is  associated  with  slack  variables  introduced  into 


B 


where  the  columns  of 


-J  P 


r-i 


-B* 

b* 


■-A^  l' 
bT  0 


-B* 

b*  -  p' 

are  selected  from  the  columns  of 


'tt. 


.  We  also  assume  that  we  have  a  (  and  0  determined 


from  the  previous  solution  of  (14). 

Now,  at  the  start  of  the  modified  simplex  technique,  we 

replace,  in  Bl>  the  column 
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initial  replacement  eliminates  the  problem  of  the  additional  con¬ 
straint  in  (12)  and  yields  a  new  basis  for  (14)  as  we  will  now  show. 
(14)  may  be  written  in  the  form: 


(16) 


where 


tT 

-J.  P 
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max  0 
-B* 

b*  - 

N/"  is  composed  of  elements  of  and  the  slack  variables  correspond¬ 
ing  to  the  basis  columns.  But  we  may  write: 

'-B* 


b* 


V 


/  prN 


+  0 


V 


b*  1 

In  order  to  show  that  is  a  basis  we  express 


in  terms  of 


the  previous  basis  Bj^  to  get: 


will  be  a  basis  if  and  only  if  4  0.  Now,  utilizing  the  fact 

that  the  constraints  in  (12)  may  be  written  in  the  form: 


or  7  =  (0^  1)  ^ 

If  we  assume  that  AX=b,  X>0has  a  solution  X  and 
Ay  =  0,  y  i  0  has  a  solution  y  0 ,  then,  for  all  ^  0 , 

A(X  +c<  y)  =  b,  X  +0^9  >  0.  But  this  contradicts  our  assumption 
that  the  convex  set  {  x|a  X  =  b ,  X  >  0  J  is  bounded.  Hence 

Ay  «=  0,  y  >  0  if  and  only  if  y  =  0.  So  we  will  not  get =0  in 

(12).  Therefore  6,  and  B],  is  a  basis  for  (14).  We  get  (auto¬ 
matically)  a  starting  feasible  solution  (  o)  from  which 

we  proceed  with  the  modified  simplex  technique. 

The  solution  of  the  Dantzig-Wolf e  auxiliary  problem  yield 
an  X,  say  X*,  which  in  turn  yields  a  P*  =  A  X*  as  the  VCI.  The 

VGO  in  the  Dantzig  -  Wolfe  technique  is  determined  after  the  aux¬ 

iliary  problem  has  been  solved.  In  our  dual  technique  the  VGO  is 
determined  at  the  beginning  of  the  iteration.  The  solution  of  our 
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auxiliary  problem,  given  by  (15),  yields  a  solution  )  bo  (12) 

and,  hence,  a  P  =  ^  as  the  VCI.  As  we  noted  before,  ^>0. 

We  make  no  direct  use  of  the  X's  in  our  auxiliary  prob¬ 
lem  as  do  Dantzig  and  Wolfe  in  the  direct  technique.  However  y  =  ■??  X  . 
X  is  an  extreme  point  of|^x|AX==  b,  X^  o'J  .  For,  if  it  were 
not,  then  there  would  be  no  lower  bound  for  y  and  the 

above  convex  set  would  not  be  bounded,  contrary  to  our  assumption. 

In  the  development  of  the  dual  decomposition  technique  no 
mention  was  made  of  the  structure  of  the  matrix  A.  However ,  in  the 
direct  decomposition  principle  the  composition  of  the  current  basis 
matrix  B  changes  as  the  structure  of  A  changes.  For  example,  for 
a  matrix  A  of  the  form: 

Ai  0  0 


A  = 


0 

0 


0 


0 


the  current  basis  matrix  is  no  longer  given  by  an  expression  com¬ 
parable  to  (6)  but  assumes  the  form: 

Plki  **21  ^22-  • 

1  1  ...  1  0  0  .  . 

0  0  ...0  1  l..,l 

0  0  ...0  0  0...0 


B  = 


^11  ^12  *  • 


P2k2  ^31  P32  '  •  •  P3k3 

0  0  ...  0 


0 


0 

1 


0 

1 


.  .  .  0 
.  .  .  1 
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In  the  dual  technique  the  structure  of  B  will  essentially  remain 
unchanged  as  the  structure  of  A  changes. 

In  summary,  our  dual  technique  is  essentially  the  modified 
simplex  technique,  as  was  stated  before,  with  the  exception  that  the 
column  to  enter  the  current  basis  is  obtained  by  solving  an  auxiliary 
problem  given  by  (14).  By  means  of  one  artificial  iteration  we  are 
able  to  eliminate  the  presence  of  the  additional  constraint  in  the 
dual  to  (14)  and  obtain  an  initial  feasible  solution  for  the  auxiliary 
problem  while,  at  the  same  time,  taking  maximx  n  benefit  of  the  previous 
computations . 

Assuming  non-degeneracy  the  iterative  process  is  finite 
since  the  number  of  extreme  points  of  the  convex  set  j  X 


I  AX  =  b ,  X  >  o| 


is  finite. 
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